Pitch and Alignment in the Perception of Tone and Intonation: Pragmatic Signals and Biological Codes

In this paper a case is made for examining tone perception and particularly phenomena of tonal alignment in terms of perceptual limitations to pitch processing. The perception of pitch and tonal movement is discussed in relationship to research on alignment. Results from perception experiments using Swedish listeners and non-scripted speech production data from Swedish speakers are used to exemplify the discussion. The results of these experiments are further discussed in terms of biological codes. Finally thresholds of tonal movement perception are proposed based on principles of syllable alignment consistent with perceptual constraints including cognitive processing. 1. Perceptual constraints in tonal processing There is currently considerable interest in examining the relationship between tone and intonation in diverse languages of the world. With its lexical tone inventory and long history as a language studied by scholars, Standard Chinese is a language well suited for investigating tonal phenomenon and the interactions between tone and intonation. Professor Zongji Wu, whom we honour with this symposium, has carried out much pioneering and inspirational work on this challenging topic [1]. In his analysis, he sees the surface realization of intonation in Standard Chinese as a result of the interaction of three components. Two of the components make up basic tone units. These are the original mono-syllabic toneme patterns and the poly-syllabic phrasal tone sandhi. The third component is sentential intonation reflecting declarative, interrogative or exclamatory intonation. The intonation component modifies the global contour and can also be changed by different speaker attitudes. He sees intonation as changing the register of the global tonal contour in much the same way as a change of key in music. The analysis of intonation as a result of the interaction of underlying components as formulated by Professor Zongji Wu helps us understand the tonal variation found in production studies for both scripted and non-scripted databases [2]. This kind of analysis is also useful in the generation of the F0 contour in speech synthesis, especially as we try to expand speech synthesis to include expressive speech under a variety of emotional situations. The complexity of intonation especially in non-scripted expressive speech also gives rise to questions regarding the perception of intonation. What are the important mechanisms and interactions involved in perceiving the multiple layers of information carried by the F0 contour? This contribution presents a perceptual account of tonal movements in an attempt to offer possible explanations of mechanisms involved in the temporal alignment of tone and intonation. The guiding principle behind this approach is the idea that incorporating biological constraints in our understanding of the speech communication process can give us insights into why speech sounds have developed as they have and can also give us an explanatory tool to help our investigation of specific sound contrasts in different languages. This type of approach to experimental phonetics has been strongly argued for during the past twenty years particularly in the works of Lindblom [3] and Ohala [4] and has been recently represented in terms of intonation and tone by Gussenhoven [5] and Xu [6]. In a larger framework, this approach takes into consideration biological constraints in the areas of motor control (speech production and articulation), the human auditory system, (speech perception), cognition and the brain (speech planning and speech understanding), and the constraints and needs of social interaction. The point of departure for perceptual constraints in tonal processing builds on earlier work on selective perception [7]. In this view, the biological resources available for perceptual processing of the speech signal are limited and must therefore be allocated to various tasks in a restricted manner. In terms of tonal processing the resources must be focussed selectively on the tasks of analysing the spectrum and extracting pitch from the incoming signal. The way in which these resources are allocated in time can be one of the important factors which can contribute to the shaping of patterns of tonal movement in speech. We can speculate that much of the tonal movement patterns we find in speech facilitate their perception in the same way as they are adapted to laryngeal gesture constraints in production. 2. Temporal alignment of tone One of the more exciting and expanding issues of tonal research during the past few decades has been the question of tonal alignment. This generally refers to the time alignment of fundamental frequency excursions in relationship to segmental and syllable boundaries and has been studied from an acoustic analysis standpoint, a production perspective and a perception point of view. An important question concerning tonal alignment has been the question of variability and phonological categories. Motivated by a rule-based generative approach to intonation, the alignment of rises and falls was explored and this alignment was found to give different intonational meanings [8] and word accents [9]. These properties were first primarily investigated for pitch accent languages [10][11] but now include such issues as declarative-interrogative intonation [12] models of F0 generation and alignment [13] lexical tone alignment properties in Mandarin [14] and in the Chinese dialect of Fuzhou [15], and in Thai [16], and discourse-related intonational features [17]. In all these studies, there is a International Symposium on Tonal Aspects of Languages: With Emphasis on Tone Languages Beijing, China, March 28-31, 2004 ISCA Archive http://www.isca-speech.org/archive